program committee
ReviewerToo: Should AI Join The Program Committee? A Look At The Future of Peer Review
Sahu, Gaurav, Larochelle, Hugo, Charlin, Laurent, Pal, Christopher
Peer review is the cornerstone of scientific publishing, yet it suffers from inconsistencies, reviewer subjectivity, and scalability challenges. We introduce ReviewerToo, a modular framework for studying and deploying AI-assisted peer review to complement human judgment with systematic and consistent assessments. ReviewerToo supports systematic experiments with specialized reviewer personas and structured evaluation criteria, and can be partially or fully integrated into real conference workflows. We validate ReviewerToo on a carefully curated dataset of 1,963 paper submissions from ICLR 2025, where our experiments with the gpt-oss-120b model achieves 81.8% accuracy for the task of categorizing a paper as accept/reject compared to 83.9% for the average human reviewer. Additionally, ReviewerToo-generated reviews are rated as higher quality than the human average by an LLM judge, though still trailing the strongest expert contributions. Our analysis highlights domains where AI reviewers excel (e.g., fact-checking, literature coverage) and where they struggle (e.g., assessing methodological novelty and theoretical contributions), underscoring the continued need for human expertise. Based on these findings, we propose guidelines for integrating AI into peer-review pipelines, showing how AI can enhance consistency, coverage, and fairness while leaving complex evaluative judgments to domain experts. Our work provides a foundation for systematic, hybrid peer-review systems that scale with the growth of scientific publishing.
Accept More, Reject Less: Reducing up to 19% Unnecessary Desk-Rejections over 11 Years of ICLR Data
Li, Xiaoyu, Song, Zhao, Zhang, Jiahao
The explosive growth of AI research has driven paper submissions at flagship AI conferences to unprecedented levels, necessitating many venues in 2025 (e.g., CVPR, ICCV, KDD, AAAI, IJCAI, WSDM) to enforce strict per-author submission limits and to desk-reject any excess papers by simple ID order. While this policy helps reduce reviewer workload, it may unintentionally discard valuable papers and penalize authors' efforts. In this paper, we ask an essential research question on whether it is possible to follow submission limits while minimizing needless rejections. We first formalize the current desk-rejection policies as an optimization problem, and then develop a practical algorithm based on linear programming relaxation and a rounding scheme. Under extensive evaluation on 11 years of real-world ICLR (International Conference on Learning Representations) data, our method preserves up to $19.23\%$ more papers without violating any author limits. Moreover, our algorithm is highly efficient in practice, with all results on ICLR data computed within at most 53.64 seconds. Our work provides a simple and practical desk-rejection strategy that significantly reduces unnecessary rejections, demonstrating strong potential to improve current CS conference submission policies.
Considering Conference Contributions
Saurabh Bagchi Everything You Always Wanted to Know About PCs, But Were Afraid to Ask https://bit.ly/4e4JoAx Okay, PCs in the title could be Political Correctness or Personal Computers or even Peace Corps. It stands for Program Committees. As researchers, in academia or industry, we often are asked to serve on Program Committees of conferences in our fields of expertise. Serving on PCs signals one is a good citizen of our global technical village and has its own altruistic rewards.
Has the Machine Learning Review Process Become More Arbitrary as the Field Has Grown? The NeurIPS 2021 Consistency Experiment
Beygelzimer, Alina, Dauphin, Yann N., Liang, Percy, Vaughan, Jennifer Wortman
We present the NeurIPS 2021 consistency experiment, a larger-scale variant of the 2014 NeurIPS experiment in which 10% of conference submissions were reviewed by two independent committees to quantify the randomness in the review process. We observe that the two committees disagree on their accept/reject recommendations for 23% of the papers and that, consistent with the results from 2014, approximately half of the list of accepted papers would change if the review process were randomly rerun. Our analysis suggests that making the conference more selective would increase the arbitrariness of the process. Taken together with previous research, our results highlight the inherent difficulty of objectively measuring the quality of research, and suggest that authors should not be excessively discouraged by rejected work.
Google at ECCV 2022
Google is proud to be a Platinum Sponsor of the European Conference on Computer Vision (ECCV 2022), a premier forum for the dissemination of research in computer vision and machine learning (ML). This year, ECCV 2022 will be held as a hybrid event, in person in Tel Aviv, Israel with virtual attendance as an option. Google has a strong presence at this year's conference with over 60 accepted publications and active involvement in a number of workshops and tutorials. We look forward to sharing some of our extensive research and expanding our partnership with the broader ML research community. We hope you'll visit our on-site or virtual booths to learn more about the research we're presenting at ECCV 2022, including several demos and opportunities to connect with our researchers.
ICPRAM 2021 Conference Report
ICPRAM 2021 (10th International Conference on Pattern Recognition Applications and Methods) received 97 paper submissions from 30 countries. To evaluate each submission, a double‐blind paper review was performed by the Program Committee. After a stringent selection process, 21 papers were published and presented as full papers, i.e. completed work (12 pages/25' oral presentation), 53 papers were accepted as short papers (28 as oral presentation and 25 as poster presentation). ICPRAM's program included three invited talks delivered by internationally distinguished speakers, namely: The papers were organized in thirteen parallel sessions ranging from areas such as Machine Learning Methods; Deep Learning and Neural Networks; Classification and Clustering; Natural Language Processing; Theory and Methods; Methods and Applications; and Image and Video Analysis and Understanding. The organizing committee included the ICPRAM Conference Chair: Ana Fred, Instituto de Telecomunicações and University of Lisbon, Portugal; and the Program Co‐Chairs: Maria De Marsico, Sapienza Università di Roma, Italy; and Gabriella Sanniti di Baja, Italian National Research Council CNR, Italy.
Collusion Rings Threaten the Integrity of Computer Science Research
The discipline of computer science has historically made effective use of peer-reviewed conference publications as an important mechanism for disseminating timely and impactful research results. Recent attempts to "game" the reviewing system could undermine this mechanism, damaging our ability to share research effectively. I want to alert the community to a growing problem that attacks the fundamental assumptions that the review process has depended upon. My hope is that exposing the behavior of a community of unethical individuals will encourage others to exert social pressure that will help bring colluders into line, invite a broader set of people to engage in problem solving, and provide some encouragement for people trapped into collusion by more senior researchers to extricate themselves and make common cause with the rest of the community. My motivation for writing this Viewpoint is because I became aware of an example in the computer-architecture community where a junior researcher may have taken his own life instead of continuing to engage in a possible collusion ring.a Collusion rings extend far beyond the field of computer architecture.
SysML: The New Frontier of Machine Learning Systems
Ratner, Alexander, Alistarh, Dan, Alonso, Gustavo, Andersen, David G., Bailis, Peter, Bird, Sarah, Carlini, Nicholas, Catanzaro, Bryan, Chayes, Jennifer, Chung, Eric, Dally, Bill, Dean, Jeff, Dhillon, Inderjit S., Dimakis, Alexandros, Dubey, Pradeep, Elkan, Charles, Fursin, Grigori, Ganger, Gregory R., Getoor, Lise, Gibbons, Phillip B., Gibson, Garth A., Gonzalez, Joseph E., Gottschlich, Justin, Han, Song, Hazelwood, Kim, Huang, Furong, Jaggi, Martin, Jamieson, Kevin, Jordan, Michael I., Joshi, Gauri, Khalaf, Rania, Knight, Jason, Konečný, Jakub, Kraska, Tim, Kumar, Arun, Kyrillidis, Anastasios, Lakshmiratan, Aparna, Li, Jing, Madden, Samuel, McMahan, H. Brendan, Meijer, Erik, Mitliagkas, Ioannis, Monga, Rajat, Murray, Derek, Olukotun, Kunle, Papailiopoulos, Dimitris, Pekhimenko, Gennady, Rekatsinas, Theodoros, Rostamizadeh, Afshin, Ré, Christopher, De Sa, Christopher, Sedghi, Hanie, Sen, Siddhartha, Smith, Virginia, Smola, Alex, Song, Dawn, Sparks, Evan, Stoica, Ion, Sze, Vivienne, Udell, Madeleine, Vanschoren, Joaquin, Venkataraman, Shivaram, Vinayak, Rashmi, Weimer, Markus, Wilson, Andrew Gordon, Xing, Eric, Zaharia, Matei, Zhang, Ce, Talwalkar, Ameet
Machine learning (ML) techniques are enjoying rapidly increasing adoption. However, designing and implementing the systems that support ML models in real-world deployments remains a significant obstacle, in large part due to the radically different development and deployment profile of modern ML methods, and the range of practical concerns that come with broader adoption. We propose to foster a new systems machine learning research community at the intersection of the traditional systems and ML communities, focused on topics such as hardware systems for ML, software systems for ML, and ML optimized for metrics beyond predictive accuracy. To do this, we describe a new conference, SysML, that explicitly targets research at the intersection of systems and machine learning with a program committee split evenly between experts in systems and ML, and an explicit focus on topics at the intersection of the two.
The Fourth International Conference on Informatics in Control, Automation, and Robotics (ICINCO 2007)
The Fourth International Conference on Informatics in Control, Automation, and Robotics (ICINCO 2007) was held at the University of Angers from 9 through 12 May 2007. This conference sought to bring together researchers, engineers, and practitioners interested in the application of informatics to control, automation, and robotics, with an emphasis on intelligent systems and various AI technologies, such as expert systems, evolutionary computing, neural networks, and others, in connection to signal processing, systems modeling, and control. Beside the presentation of papers addressing these general topics, several specific themes were discussed during the conference in specialized forums, including special sessions, panels, and workshops, as described in this report. The conference was coorganized by the Institute for Systems and Technologies of Information, Control, and Communication (INSTICC) and the University of Angers, through the Laboratoire d'Ingénierie des Systèmes AI Magazine Volume 28 Number 4 (2007) ( AAAI) The conference was also held in cooperation with the Association for the Advancement of Artificial Intelligence (AAAI). The ICINCO 2007 conference program included oral presentations (full papers and short papers), as well as posters, organized in three simultaneous tracks: "Intelligent Control Systems and Optimization," "Robotics and Automation," and "Systems Modeling, Signal Processing, and Control."